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LISTING OF THE CLAIMS 

Claims 1-56 are pending. No claims are amended, added, or canceled. 

The following listing of claims replaces all prior versions and listings of claims in 
the application. 

1. (Original) A method for image retrieval using a statistical bigram correlation 
model, the method comprising: 

receiving a plurality of images responsive to multiple search sessions; 

determining whether the images are semantically relevant images via 
relevance feedback; and 

estimating a respective semantic correlation between each of at least one 
pair of the images with a respective bigram frequency, each respective bigram 
frequency being based on multiple search sessions in which each image of the pair 
is indicated to be a semantically relevant image. 

2. (Original) A method as recited in claim 1 , further comprising: 

assigning a respective ranking score to each of the images based at least in 
part on the respective semantic correlation corresponding to the image; and 
displaying only those images with a highest range of ranking scores. 

3* (Original) A method as recited in claim 1, further comprising, responsive to 
a search session, dynamically updating the respective bigram frequency 
corresponding to two of the images. 
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4. (Original) A method as recited in claim 1, wherein the respective semantic 
correlation is: (a) a positive correlation between two semantically relevant images; 
(b) a negative correlation between a semantically relevant image and a 
semantically irrelevant image; and (c) no correlation otherwise. 

5. (Original) A method as recited in claim 1 : 
wherein the respective semantic correlation is performed offline or online 

to calculate unigram and bigram frequencies from relevance feedback information, 
the unigram frequency being based on relevance feedback to a session of the 
multiple search sessions, the unigram frequency indicating that each respective 
image of the images is either semantically relevant to the session, semantically 
less relevant to the session, or a non-feedback image with respect to the session; 
and 

wherein each respective bigram frequency is based on a pair of unigram 
frequencies. 
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17 6. (Original) A method as recited in claim 1, wherein estimating the respective 
is semantic correlation further comprises: 

19 associating a respective unigram frequency with each of the images, the 

20 unigram frequency indicating that each respective image of the images is either 

21 semantically relevant, semantically less relevant, or a non-feedback image, the 
n unigram frequency being based on relevance feedback to a session of the multiple 

23 search sessions; and 

24 wherein each respective bigram frequency is based on a pair of unigram 
2s frequencies. 
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7. (Original) A method as recited in claim 1, wherein estimating the respective 
semantic correlation further comprises: 

associating a respective unigram frequency with each of the images, the 
unigram frequency indicating that each respective image of the images is either 
semantically relevant, semantically less relevant, or a non-feedback image, the 
unigram frequency being based on relevance feedback to a session of the multiple 
search sessions; 

determining a maximum frequency from a maximum value of the bigram 
and unigram frequencies; and 

wherein the respective semantic correlation is further based on the 
maximum frequency. 

8* (Original) A method as recited in claim 1, further comprising identifying, for 
each image obtained responsive to one or more search sessions of the multiple 
search sessions, a respective semantic support based on a similarity measure 
and/or the respective semantic correlation, the similarity measure corresponding to 
a similarity of a respective feature vector of the image and a search query 
corresponding to the session. 

9. (Original) A method as recited in claim 1, further comprising: 

identifying, for each image obtained responsive to one or more search 
sessions of the multiple search sessions, a respective semantic support based on a 
similarity measure and/or the respective semantic correlation, the similarity 
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measure corresponding to a similarity of a respective feature vector of the image 
and a search query corresponding to the session; 

assigning a respective ranking score to each of the images based upon the 
respective similarity measure, the respective semantic support, and a semantic 
weight; and 

displaying only those images with a highest range of respective Tanking 

scores. 

9 10. (Original) A method as recited in claim 1, wherein estimating the 

10 respective semantic correlation is determined as follows: 
is • 0 < R(I,J) < 1 [if is true in all cases] 

12 ♦ R(I,S) = R(J,J) ; [if is true in all cases] 

13 • if /^and £/(/) < 0 ; R(I,J) = 0 ; 

14 • if / * J and B(I 9 J) < 0 : R(I 9 J) = 0 ; 
is • if I=J*md U(I) > 0 : R{l 9 j) = U(I)/T ; or 
M • if / * J and B(I,J) > 0 : R(l,j)= B(l)/T ; 

17 wherein -/are two images* B(I, J) is their bigram frequency, is the 

11 unigram frequency of image I,T is the maximum frequency, R(I,J) is the 
\9 correlation between image I and J. 

20 

si 11. (Original) A method as recited in claim I, wherein each respective bigram 

22 frequency is based on a pair of unigram frequencies, and wherein the method 

23 further comprises performing the respective semantic correlation offline by: 

24 (a) initializing all unigram and bigram frequencies to zero; 

25 (b) clustering search sessions with a same query into groups; 
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(c) calculating unigram counts within a group; 

(d) updating unigram frequencies ; 

(e) updating bigram frequencies; 

(f) repeating operations (c), (d), and (f) for all session groups; 

(g) setting all negative unigram and bigram frequencies to zero; and 

(h) calculating each respective semantic correlation based on results of (a) 
through (f). 

12. (Original) A method as recited in claim l s wherein each respective bigram 
frequency is based on a pair of unigram frequencies, wherein C(I) is a unigram 
count of image /, and wherein the method further comprises performing the 
respective semantic correlation offline by: 

(a) initializing C(I) to zero (0); 

(b) itcratively updating C(I) for every session in a group such that: 

C(7) = C(7) + 1 , if image / is labeled as relevant in a session; 
C(7) = C(7)-1, if image / is labeled as irrelevant in a session; 

and 

C(7) is unchanged otherwise. 

(c) repeating (b) for every image of the images; 

(d) updating each respective unigram frequencies as U(I) = U(I) + C(I) ; 

(e) updating each respective bigram frequency of an image pair such 



that: 



B{I,J) = 5(7, J) + mw{C(I),C(J)} , if C(7) > 0,C(J) > 0, 
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B(I,J) = 5(7, J) - min{C(/) f -C(y)} , if C(7) > 0,C(J) < 0 S 

B(I,J) = B(I 9 J) - min{«"C(/),C(J)} , if C(7) < 0,C(*/) > 0, and 

B(I,J) = B(J,J) > otherwise; and 

wherein /, ^ are two images, B(I, J) i$ their bigram frequency, and 
U(I) is the unigram frequency of image /. 

13* (Original) A method as recited in claim 1, wherein each respective bigram 
frequency is based on a pair of unigram frequencies, and wherein the method 
flirther comprises performing the respective semantic correlation online by: 

(a) calculating unigram counts in a particular search session; 

(b) updating unigram frequencies; 

(c) updating bigram frequencies; and 

(d) updating each respective semantic correlation between each of the 
images based on results of (a) through (c). 
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14. (Original) A method as recited in claim 1, wherein each respective bigram 
frequency is based on a pair of unigram frequencies, wherein C(7) is a unigram 
count of image /, wherein U(I) is a unigram frequency of image 7, wherein 
B(I,J) is a bigram frequency of image 7 and J, wherein a session group comprises 
a single search session, and wherein the method further comprises performing the 
respective semantic correlation online by: 

(a) responsive to determining that there is a user log, updating 
calculating each respective unigram and bigram frequency according to data in 
the user log; 

(b) responsive to determining that there is not a user log, initializing 
each C(I) and B(l) to zero (0); 

(c) iteratively updating C(I) for the single search session such that: 

C(7) as 1 , if image / is labeled as relevant; 

C(7) = -1 , if image 7 is labeled as irrelevant; and 

C(7) = 0 , if C(I) is a non-feedback image; 

(d) updating each respective unigram frequencies as U(I) = U(I) + C(7) ; 

(e) updating each respective bigram frequency of an image pair such 

that: 

B(I,J) - B(I 9 J)+l , if C(7) > 0,C(J) > 0, 
5(7, J) = B(I f J) - 1 f if C(7) > 0.CG7) < 0, 
5(7, J) = B(I,J) - 1 , if C(I) < 0, C(J) > 0, or 
B(I 9 J) = B(I,J) 9 otherwise; and 
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wherein /. J are two images, and B(I, J) is their bigram frequency. 

15. (Original) A computer-readable medium for image retrieval using a 
statistical bigram correlation model, the computer-readable medium comprising 
computer-executable instructions for: 

receiving a plurality of images responsive to multiple search sessions; 

determining whether the images are semantically relevant images via 
relevance feedback; and 

estimating a respective semantic correlation between each of at least one 
pair of the images with a respective bigram frequency, each respective bigram 
frequency representing a probability of whether two of the images are 
semantically related to one-another based on a co-occurrence frequency that each 
image of the two images was relevant in a previous query/feedback session. 

16. (Original) A computer-readable medium as recited in claim 15, further 
comprising instructions fon 

assigning a respective ranking score to each of the images based at least in 
part on the respective semantic correlation corresponding to the image; and 
displaying only those images with a highest range of ranking scores. 

17. (Original) A computer-readable medium as recited in claim 15, further 
comprising instructions for, responsive to a search session, dynamically updating 
the respective bigram frequency corresponding to two of the images* 
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18. (Original) A computer-readable medium as recited in claim 1 5, wherein the 
respective semantic correlation is: (a) a positive correlation between two 
semantically relevant images; (b) a negative correlation between a semanticaUy 
relevant image and a semantically irrelevant image; and (c) no correlation 
otherwise. 

19. (Original) A computer-readable medium as recited in claim 15: 

wherein the respective semantic correlation is performed offline or online 
to calculate unigram and bigram frequencies from relevance feedback information, 
the unigram frequency being based on relevance feedback to a session of the 
multiple search sessions, the unigram frequency indicating that each respective 
image of the images is either semantically relevant to the session, semantically 
less relevant to the session, or a non-feedback image with respect to the session; 
and 

wherein each respective bigram frequency is based on a pair of unigram 
frequencies. 



20, (Original) A computer-readable medium as recited in claim 15, wherein 
estimating the respective semantic correlation further comprises instructions for: 

20 associating a respective unigram frequency with each of the images, the 

21 unigram frequency indicating that each respective image of the images is either 

22 semantically relevant, semantically less relevant, or a non-feedback image, the 

23 unigram frequency being based on relevance feedback to a session of the multiple 

24 search sessions; and 
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wherein each respective bigram frequency is based on a pair of unigram 
frequencies. 

21. (Original) A computer-readable medium as recited in claim 15, wherein 
estimating the respective semantic correlation further comprises instructions for: 

associating a respective unigram frequency with each of the images, the 
unigram frequency indicating that each respective image of the images is either 
semantically relevant, semantically less relevant, or a non-feedback image, the 
unigram frequency being based on relevance feedback to a session of the multiple 
search sessions; 

determining a maximum frequency from a maximum value of the bigram 
and unigram frequencies; and 

wherein the respective semantic correlation is further based on the 
maximum frequency, 

22, (Original) A computer-readable medium as recited in claim 15, further 
comprising instructions for identifying, for each image obtained responsive to one 
or more search sessions of the multiple search sessions, a respective semantic 
support based on a similarity measure and/or the respective semantic correlation, 
the similarity measure corresponding to a similarity of a respective feature vector 
of the image and a search query corresponding to the session. 
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23. (Original) A computer-readable medium as recited in claim 15, further 
comprising instructions for: 

identifying* for each image obtained responsive to one or more search 
sessions of the multiple search sessions, a respective semantic support based on a 
similarity measure and/or the respective semantic correlation, the similarity 
measure corresponding to a similarity of a respective feature vector of the image 
and a search query corresponding to the session; 

assigning a respective ranking score to each of the images based upon the 
respective similarity measure, the respective semantic support, and a semantic 
weight; and 

displaying only those images with a highest range of respective ranking 

scores. 

24. (Original) A computer-readable medium as recited in claim 15, wherein 
estimating the respective semantic correlation is determined as follows: 

• 0 <;*(/, J) £ 1 

• R(I,J) = R(J y J); 

• if I=Jand U(I) £ 0 : /?(/, J) = 0 ; 

• if I * J and B(I, J) <0 : R(I,J) = 0; 

• if 7=7and £/(/) > 0: R(l,j)=U(I)/T;or 

• if / * J and B(I 3 J) > 0 : R(l,j)= B(I)/T ; 

wherein I, J are two images, B(I 9 J) is their bigram frequency, U(I) is the 
unigram frequency of image I,T is the maximum frequency, R(I*J) is the 
correlation between image I and J. 
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25. (Original) A computer-readable medium as recited in claim 15, wherein 
each respective bigram frequency is based on a pair of unigram frequencies, and 
wherein the computer-executable instructions ftirther comprise instructions for 
performing the respective semantic correlation offline by: 

(a) initializing all unigram and bigram frequencies to zero; 

(b) clustering search sessions with a same query into groups; 

(c) calculating unigram counts within a group; 

(d) updating unigram frequencies ; 

(e) updating bigram frequencies; 

(f) repeating operations (c), (d) f and (f) for all session groups; 

(g) setting all negative unigram and bigram frequencies to zero; and 

(h) calculating each respective semantic correlation based on results of (a) 
through (f). 

26. (Original) A computer-readable medium as recited in claim 15, wherein 
each respective bigram frequency is based on a pair of unigram frequencies, 
wherein C(/) is a unigram count of image I and wherein the computer-executable 
instructions further comprise instructions for performing the respective semantic 
correlation offline by; 

(a) initializing C(I) to zero (0); 

(b) iteratively updating C(I) for every session in a group such that; 

C(7) - C(7) + 1 , if image / is labeled as relevant in a session; 
C(/) = C(/)-l, if image / is labeled as irrelevant in a session; 

and 
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C(7) is unchanged otherwise; 

(c) repeating (b) for every image of the images; 

(d) updating each respective unigram frequencies as U{I) = U(I) + C(7) ; 

(e) updating each respective bigram frequency of an image pair such 

that: 

B(I,J) = 5(7,7) + min{C(7) 4 C(7)} , if C(7) > 0,C(7) > 0, 

5(7, 7) = 5(7,7) - rain{C(7),-C(7)} , if C(J) > 0,C(7) < 0, 

5(7,7) = 5(7,7) - min{-C(7),C(7)} , if C(/) < 0,C(7) > 0, and 

5(7,7) = 5(7,7) , otherwise; and 

wherein 7, J are two images, 5(7,7) is their bigram frequency, and 
is the unigram frequency of image 7, 

27. (Original) A computer-readable medium as recited in claim 15, wherein 
each respective bigram frequency is based on a pair of unigram frequencies, and 
wherein the computer-executable instructions further comprise instructions for 
performing the respective semantic correlation online by: 

(a) calculating unigram counts in a particular search session; 

(b) updating unigram frequencies; 

(c) updating bigram frequencies; and 

(d) updating each respective semantic correlation between each of the 
images based on results of (a) through (c). 
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28. (Original) A computer-readable medium as recited in claim 15, wherein 
each respective bigram frequency is based on a pair of unigram frequencies, 
wherein C(7) is a unigram count of image /, wherein £/(/) is a unigram frequency 
of image /, wherein B{I,J) is a bigram frequency of image / and J, wherein a 
session group comprises a single search session, and wherein the computer- 
executable instructions further comprise instructions for performing the respective 
semantic correlation online by: 

(a) responsive to determining that there is a user log, updating 
calculating each respective unigram and bigram frequency according to data in 
the user log; 

(b) responsive to determining that there is not a user log, initializing 
each C(T) and B(I) to zero (0); 

(c) iteratively updating C(I) for the single search session such that: 

CCO = 1 . if image / is labeled as relevant; 

C(I) = -1 s if image / is labeled as irrelevant; and 

C(l) = 0 , if C(I) is a non-feedback image; 

(d) updating each respective unigram frequencies as U(I) = U(I) + C(7) ; 

(e) updating each respective bigram frequency of an image pair such 

that: 

B{I,J) = 5(/ 5 J) + 1 , if C(/) > 0,C( J) > 0, 
B(I,J) = B(I,J) - 1 , if C(I) > Q,C(J) < 0, 
B(I,J) = B(I 9 J) - 1 , if C(/) < 0,C(J) > 0, or 
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B(I 9 J) = B{I t J) , otherwise; and 
wherein /, J are two images, and B(I, J) is their bigram frequency. 

29. (Original) A computing device for image retrieval using a statistical bigram 
correlation model, the computing device comprising: 

a processor; and 

a memory coupled to the processor, the memory comprising computer- 
executable instructions that are fetched and executed by the processor for: 

receiving a plurality of images responsive to multiple search 

sessions; 

determining whether the images are semantically relevant images via 
relevance feedback; and 

estimating a respective semantic correlation between each of at least 
one pair of the images with a respective bigram frequency, each respective bigram 
frequency being based on multiple search sessions in which each image of the pair 
is indicated to be a semantically relevant image. 

30. (Original) A computing device as recited in claim 29, fiirther comprising 
instructions for: 

assigning a respective ranking score to each of the images based at least in 
part on the respective semantic correlation corresponding to the image; and 
displaying only those images with a highest range of ranking scores. 
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31. (Original) A computing device as recited in claim 29, further comprising 
Instructions for, responsive to a search session, dynamically updating the 
respective bigram frequency corresponding to two of the images, 

32. (Original) A computing device as recited in claim 29, wherein the 
respective semantic correlation is: (a) a positive correlation between two 
semantically relevant images; (b) a negative correlation between a semantical ly 
relevant image and a semantically irrelevant image; and (c) no correlation 
otherwise. 

33. (Original) A computing device as recited in claim 29: 

wherein the respective semantic correlation is performed offline or online 
to calculate unigram and bigram frequencies from relevance feedback information, 
the unigram frequency being based on relevance feedback to a session of the 
multiple search sessions, the unigram frequency indicating that each respective 
image of the images is either semantically relevant to die session, semantically 
less relevant to the session, or a non-feedback image with respect to the session; 
and 

wherein each respective bigram frequency is based on a pair of unigram 
frequencies. 

34. (Original) A computing device as recited in claim 29, wherein estimating 
the respective semantic correlation further comprises instructions for: 

associating a respective unigram frequency with each of the images, the 
unigram frequency indicating that each respective image of the images is either 
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semantically relevant, semantically less relevant, or a non-feedback image, the 
unigram frequency being based on relevance feedback to a session of the multiple 
search sessions; and 

wherein each respective bigram frequency is based on a pair of unigram 
frequencies. 

35. (Original) A computing device as recited in claim 29, wherein estimating 
the respective semantic correlation further comprises instructions for: 

associating a respective unigram frequency with each of the images, the 
unigram frequency indicating that each respective image of the images is either 
semantically relevant, semantically less relevant, or a non-feedback image, the 
unigram frequency being based on relevance feedback to a session of the multiple 
search sessions; 

determining a maximum frequency from a maximum value of the bigram 
and unigram frequencies; and 

wherein the respective semantic correlation is further based on the 
maximum frequency. 

36. (Original) A computing device as recited in claim 29, further comprising 
instructions for identifying, for each image obtained responsive to one or more 
search sessions of the multiple search sessions, a respective semantic support 
based on a similarity measure and/or the respective semantic correlation, the 
similarity measure corresponding to a similarity of a respective feature vector of 
the image and a search query corresponding to the session. 
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1 37. (Original) A computing device as recited in claim 29, Anther comprising 

2 instructions for: 

3 identifying, for each image obtained responsive to one or more search 

4 sessions of the multiple search sessions, a respective semantic support based on a 
s similarity measure and/or the respective semantic correlation, the similarity 
$ measure corresponding to a similarity of a respective feature vector of the image 
7 and a search query corresponding to the session; 

s assigning a respective ranking score to each of the images based upon the 

9 respective similarity measure, the respective semantic support, and a semantic 

10 weight; and 

u displaying only those images with a highest range of respective ranking 

12 scores. 

13 

14 38. (Original) A computing device as recited in claim 29, wherein estimating 

is the respective semantic correlation is determined as follows; 
, 6 • 0 < R{I,J) < 1 

17 . R(I 9 J) = R(J 9 I) ; 

18 • if I=Jmd U(I) < 0 : R(I,J) = 0 ; 

19 • if / * J and B(I,J) < 0 : R(I,J) = 0 ; 

20 • if /=^and U(I) > 0 : R(l,j)=U(I)fT ; or 

21 • if / * J and #(/,*/) > 0 : R{l>j)=B(I)IT ; 

22 wherein /,J*arc two images, £(/>■/) is their bigram frequency, is the 

23 unigram frequency of image I,T is the maximum frequency, R(I,J) is the 

24 correlation between image I and J. 

25 
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39. (Original) A computing device as recited in claim 29, wherein each 
respective bigram frequency is based on a pair of unigram frequencies, and 
wherein the computer-executable instructions further comprise instructions for 
performing the respective semantic correlation offline by: 

(a) initializing all unigram and bigram frequencies to zero; 

(b) clustering search sessions with a same query into groups; 

(c) calculating unigram counts within a group; 

(d) updating unigram frequencies ; 

(e) updating bigram frequencies; 

(f) repeating operations (c), (d), and (f) for all session groups; 

(g) setting all negative unigram and bigram frequencies to zero; and 

(h) calculating each respective semantic correlation based on results of (a) 
through (f). 

40. (Original) A computing device as recited in claim 29, wherein each 
respective bigram frequency is based on a pair of unigram frequencies, wherein 
C(/) is a unigram count of image /, and wherein the computer-executable 
instructions further comprise instructions for performing the respective semantic 
correlation offline by: 

(a) initializing C(I) to zero (0); 

(b) iteratively updating C(I) for every session in a group such that: 

C(7) s C(7) + 1 , if image 1 is labeled as relevant in a session; 
C(I) = C(/)-l, if image / is labeled as irrelevant in a session; 

and 
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C(/) is unchanged otherwise; 

(c) repeating (b) for every image of the images; 

(d) updating each respective unigram frequencies as U(J) = U(I) + C(7) ; 

(e) updating each respective bigram frequency of an image pair such 



that: 



B(I,J) = B(I, J) + min{C(/),C(7)} , if C(7) > 0,C(7) > 0, 

B(I,J) - *(/,./) - min{C(T),-C(J)} , if C(7) > 0,C(J) < 0, 

5(7,7) = 6(1, J) - min{-C(/),C(J)} , if C(7) < 0,C(J) > 0, and 

B(I,J) = B(I>J) , otherwise; and 

wherein I* J axe two images, 5(7, J) is their bigram frequency, and 
U (I) is the unigram frequency of image L 

41. (Original) A computing device as recited in claim 29, wherein each 
respective bigram frequency is based on a pair of unigram frequencies, and 
wherein the computer-executable instructions further comprise instructions for 
performing the respective semantic correlation online by: 

(a) calculating unigram counts in a particular search session; 

(b) updating unigram frequencies; 

(c) updating bigram frequencies; and 

(d) updating each respective semantic correlation between each of the 
images based on results of (a) through (c). 
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42. (Original) A computing device as recited in claim 29, wherein each 
respective bigram frequency is based on a pair of unigram frequencies, wherein 
C(7) is a unigram count of image I, wherein U(I) is a unigram frequency of 
image /, wherein B(I>J) is a bigram frequency of image / and J 9 wherein a session 
group comprises a single search session, and wherein the computer-executable 
instructions further comprise instructions for performing the respective semantic 
correlation online by: 

(a) responsive to determining that there is a user log, updating 
calculating each respective unigram and bigram frequency according to data in 
the user log; 

(b) responsive to determining that there is not a user log, initializing 
each C(I) and B(I) to zero (0); 

(c) iteratively updating C(I) for the single search session such that: 

C(/) = 1 , if image / is labeled as relevant; 

C(7) - -1 , if image I is labeled as irrelevant; and 

C(/) « 0 , if C(I) is a non-feedback image; 

(d) updating each respective unigram frequencies as U(J) ■» U(I) + C(7) ; 

(e) updating each respective bigram frequency of an image pair such 

that: 

B(I 9 J) = 5(7, J) +1 , if C(7) > 0,C(J) > 0, 
B(I,J) = B{I,J) - 1 , if C(7) > 0 9 C(J) < 0, 
B(I f J) m B(I,J)- 1 , if C(7) < 0,C(J) > 0, or 
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5(7, J) m 3(I 7 J) , otherwise; and 
wherein /, / are two images, and B(I,J) is their bigram frequency. 

43. (Original) A computing device image retrieval using a statistical bigram 
correlation model, the computing device comprising: 

processing means for: 

receiving a plurality of images responsive to multiple search 

sessions; 

determining whether the images are semantically relevant images via 
relevance feedback; and 

estimating a respective semantic correlation between each of at least 
one pair of the images with a respective bigram frequency, each respective bigram 
frequency being based on multiple search sessions in which each image of the pair 
is indicated to be a semantically relevant image. 

44. (Original) A computing device as recited in claim 43, further comprising 
means for: 

assigning a respective ranking score to each of the images based at least in 
part on the respective semantic correlation corresponding to the image; and 
displaying only those images with a highest range of ranking scores. 

45. (Original) A computing device as recited in claim 43, Anther comprising 
means for, responsive to a search session, dynamically updating the respective 
bigram frequency corresponding to two of the images. 
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46. (Original) A computing device as recited in claim 43, wherein the 
respective semantic correlation is: (a) a positive correlation between two 
semantically relevant images; (b) a negative correlation between a semantically 
relevant image and a semantically irrelevant image; and (c) no correlation 
otherwise. 

47. (Original) A computing device as recited in claim 43: 

wherein the respective semantic correlation is performed offline or online 
to calculate unigram and bigram frequencies from relevance feedback information, 
the unigram frequency being based on relevance feedback to a session of the 
multiple search sessions, the unigram frequency indicating that each respective 
image of the images is either semantically relevant to the session, semantically 
less relevant to the session, or a non-feedback image with respect to the session; 
and 

wherein each respective bigram frequency is based on a pair of unigram 
frequencies. 

48. (Original) A computing device as recited in claim 43, wherein the 
processing means for estimating the respective semantic correlation fUrther 
comprises means for; 

associating a respective unigram frequency with each of the images, the 
unigram frequency indicating that each respective image of the images is either 
semantically relevant, semantically less relevant, or a non-feedback image, the 



25 



PAGE 26/51 * RCVD AT 9/21/2004 6:46:23 PM [Eastern Daylight rime] * SVR:USPT0-EFXRF-1/3 * DN1S:8729306 * CS1D:3035390271 * DURATION (mra-ss):1 M6 



SEP 21 2004 16:54 FR LEE AND HAYES -PLLC 

AppLNo. 10/074,941 
. Response to Office Action dated July 21, 2004 



3035390271 TO 17038729306 



P. 27/51 



2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
IS 
19 
20 
21 
22 
23 
24 
25 



unigram frequency being based on relevance feedback to a session of the multiple 
search sessions; and 

wherein each respective bigram frequency is based on a pair of unigram 
frequencies. 

49. (Original) A computing device as recited in claim 43, wherein the 
processing means for estimating the respective semantic correlation further 
comprises means for: 

associating a respective unigram frequency with each of the images, the 
unigram frequency indicating that each respective image of the images is either 
semantically relevant, semantically less relevant, or a non-feedback image, the 
unigram frequency being based on relevance feedback to a session of the multiple 
search sessions; 

determining a maximum frequency from a maximum value of the bigram 
and unigram frequencies; and 

wherein the respective semantic correlation is further based on the 
maximum frequency. 

50. (Original) A computing device as recited in claim 43, further comprising 
processing means for identifying, for each image obtained responsive to one or 
more search sessions of the multiple search sessions, a respective semantic support 
based on a similarity measure and/or the respective semantic correlation, the 
similarity measure corresponding to a similarity of a respective feature vector of 
the image and a search query corresponding to the session. 
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51. (Original) A computing device as recited in claim 43, further comprising 
processing means for: 

identifying, for each image obtained responsive to one or more search 
sessions of the multiple search sessions, a respective semantic support based on a 
similarity measure and/or the respective semantic correlation, the similarity 
measure corresponding to a similarity of a respective feature vector of the image 
and a search query corresponding to the session; 

assigning a respective ranking score to each of the images based upon the 
respective similarity measure, the respective semantic support, and a semantic 
weight; and 

displaying only those images with a highest range of respective ranking 

scores. 

52. (Original) A computing device as recited in claim 43, wherein the 
processing means for estimating the respective semantic correlation is determined 
as follows: 

• 0 S R(I,J) £ 1- 

• R(I 9 J) = R(J 9 I); 

• if /=^and U(I) £ 0 : R(I,J) = 0 ; 

• if / * J and B{I>J) <; 0: R(I,J) = 0; 

• if I=J and U (1) > 0 : *(/, j) » U(I)/T ; or 

• if / * J and B(I 9 J) > 0 : r(i 9 j)=b(I)/T; 

wherein I, J are two images, B(I t J) i s their bigram frequency, U(J) is the 
unigram frequency of image I>T is the maximum frequency, R(I,J) is the 
correlation between image I and J. 
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53. (Original) A computing device as recited in claim 43, wherein each 
respective bigram frequency is based on a pair of unigram frequencies, and 
wherein the processing means Anther comprise means for performing the 
respective semantic correlation offline by: 

(a) initializing all unigram and bigram frequencies to zero; 

(b) clustering search sessions with a same query into groups; 

(c) calculating unigram counts within a group; 

(d) updating unigram frequencies ; 

(e) updating bigram frequencies; 

(f) repeating operations (c), (d), and (f) for all session groups; 

(g) setting all negative unigram and bigram frequencies to zero; and 

(h) calculating each respective semantic correlation based on results of (a) 
through (f). 

54. (Original) A computing device as recited in claim 43, wherein each 
respective bigram frequency is based on a pair of unigram frequencies, wherein 
C(7) is a unigram count of image /, and wherein the processing means further 
comprise means for performing the respective semantic correlation offline by: 

(a) initializing C(I) to zero (0); 

(b) iteratively updating C(I) for every session in a group such that: 

C(I) = C(/) + 1 5 if image / is labeled as relevant in a session; 
C(/) - C(/)-l, if image / is labeled as irrelevant in a session; 

and 
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C(7) is unchanged otherwise; 

(c) repeating (b) for every image of the images; 

(d) updating each respective unigram frequencies as U(I) = £7(7) + C(I) ; 

(e) updating each respective Ingram frequency of an image pair such 



that: 



B(I>J) « B(I,J) + min{C(/),C(J)} . if C(I) > 0,C(J) > 0, 

B(I,J) m B(I,J) - min{C(/),-CG/)} > if C(T) > 0,C(J) < 0, 

B{I y J) o 2?(7,*7) - min{-C(/),C(y)} , if C(/) < 0,C(7) > 0, and 

£(7, ,7) = -8(7,,/) , otherwise; and 

wherein I>J are two images, B(I, J) is their bigram frequency, and 
£7(7) is the unigram frequency of image L 

55. (Original) A computing device as recited in claim 43, wherein each 
respective bigram frequency is based on a pair of unigram frequencies, and 
wherein the processing means further comprise means for performing the 
respective semantic correlation online by: 

(a) calculating unigram counts in a particular search session; 

(b) updating unigram frequencies; 

(c) updating bigram frequencies; and 

(d) updating each respective semantic correlation between each of the 
images based on results of (a) through (c). 
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56. (Original) A computing device as recited in claim 43, wherein each 
respective bigram frequency is based on a pair of unigram frequencies, wherein 
C(I) is a unigram count of image /, wherein U(I) is a unigram frequency of 
image /, wherein 5(7, J) is a bigram frequency of image I and J 9 wherein a session 
group comprises a single search session, and wherein the processing means further 
comprise means for performing the respective semantic correlation online by: 

(a) responsive to determining that there is a user log, updating 
calculating each respective unigram and bigram frequency according to data in 
the user log; 

(b) responsive to determining that there is not a user log, initializing 
each C(J) and B(I) to zero (0); 

(c) iteratively updating C(I) for the single search session such that: 

C(7) = 1 , if image / is labeled as relevant; 

C(7) = -1 , if image / is labeled as irrelevant; and 

C(7) - 0 , if C(I) is a non-feedback image; 

(d) updating each respective unigram frequencies as 17(7) = U(I) + C(7) ; 

(e) updating each respective bigram frequency of an image pair such 



that: 



B(I 9 J) = B{I,J) + 1 , if C(7) > 0,C(J) > 0, 
B(I,J) = B(I,J) - 1 , if coo > o,C(y) < 0, 
B{I>J) m B(I,J) - 1 , if C(7) < Q,C(J) > 0, or 
5(7, J) = 5(7, J) , otherwise; and 
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wherein /» J are two images, and B(I, J) is their bigram frequency. 
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